Papers with fast inference
MCAD: Multi-teacher Cross-modal Alignment Distillation for efficient image-text retrieval (2024.findings-naacl)
Copied to clipboard
| Challenge: | Large-scale visual-language pretraining models have shown remarkable capabilities in understanding both vision and language. |
| Approach: | They propose a multi-teacher cross-modality alignment distillation technique to integrate the advantages of single-stream and dual-stream models. |
| Outcome: | The proposed model is lightweight and has only 100M running memory and 8.0ms search latency. |
FastCorrect 2: Fast Error Correction on Multiple Candidates for Automatic Speech Recognition (2021.findings-emnlp)
Copied to clipboard
Yichong Leng, Xu Tan, Rui Wang, Linchen Zhu, Jin Xu, Wenjie Liu, Linquan Liu, Xiang-Yang Li, Tao Qin, Edward Lin, Tie-Yan Liu
| Challenge: | Error correction is widely used in automatic speech recognition (ASR) to post-process the generated sentence. |
| Approach: | They propose a fast correction model that takes multiple ASR candidates as input for better correction accuracy. |
| Outcome: | The proposed model can reduce the word error rate (WER) with multiple candidates by 3.2% and 2.6%. |
Constraint-based Learning of Phonological Processes (D19-1)
Copied to clipboard
| Challenge: | Phonological processes govern the way speech sounds in natural languages change depending on context . a novel approach to learning phonological processes from related utterances is proposed . |
| Approach: | They propose an unsupervised approach to learning phonological processes from related utterances . they encode the problem into Boolean constraints that enable data efficiency and fast inference . |
| Outcome: | The proposed approach achieves high accuracy at interactive speeds on phonology problems and datasets. |
Accelerating Transformer Inference for Translation via Parallel Decoding (2023.acl-long)
Copied to clipboard
Andrea Santilli, Silvio Severino, Emilian Postolache, Valentino Maiorca, Michele Mancusi, Riccardo Marin, Emanuele Rodola
| Challenge: | Autoregressive decoding limits the efficiency of transformers for Machine Translation (MT) Existing methods to solve this problem are expensive and require changes to the model. |
| Approach: | They propose to reframe autoregressive decoding with a parallel formulation . they propose to speed up existing models without training or modifications while retaining translation quality. |
| Outcome: | The proposed model speeds up existing models without training or modifications while retaining translation quality. |
FastDiSS: Few-step Match Many-step Diffusion Language Model on Sequence-to-Sequence Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing models that correct errors in the model but lack a high quality of output . a novel training framework that matches inference noise to the model's inference signal improves performance . |
| Approach: | They propose a training framework that matches inference noise to model errors by perturbing the self-conditioning signal to match inference errors. |
| Outcome: | The proposed framework surpasses standard continuous diffusion models while providing 400x faster inference speed. |